The present invention relates to multimedia and computer graphics processing. More specifically, the present invention relates to the delivery and distributed scaling of data representing diverse media content.
As access to data networks, such as the Internet, continues to increase there is also an increased demand for the exchange of diverse multimedia content, e.g., video, audio, text, and images. Since data networks and data networking protocols were originally designed when processing power and bandwidth was relatively expensive by today's standards, these data networks and data networking protocols were designed solely for the exchange of text information. In recent years, as processing power and bandwidth has become less expensive, there has become an increased desire to exchange multimedia content which is “computationally expensive”, e.g., real-time and non-real-time digital video, real-time and non-real-time digital video coupled with synchronized audio, interactive video games and teleconferencing. However, not every user of a data communication network has equal processing power and bandwidth. To address this disparity in processing power and bandwidth, methods for scaling the multimedia data have been introduced. For example, multimedia data which in its original form would require a large amount of processing power and a large amount of bandwidth is scaled down into a format which is compatible with the processing power and bandwidth of the intended receiver of the multimedia content.
Scalability of online content is addressed by the Internet Media Initiative (IMI) established by Intel Architecture Labs. In accordance with IMI, scalable media components and applications rely on a computer processor's power to provide the highest fidelity reproduction of the original multimedia content. Accordingly, scalable content is adapted to the receiving platform's capabilities and connection bandwidth capacity. In accordance with the computer's processing power, the computer will play a scalable component at the highest level of computation that it supports. This is referred to by IMI as scalable media. There are many deficiencies with the scalable media technology in accordance with IMI. For example, IMI is directed to the scalability of already created static content, e.g., a web page or a 3D computer game, and hence, does not address scalability problems encountered in real-time communication systems. Further, IMI is directed solely to computers which operate on “PC” platforms.
Scalability of multimedia data has also been addressed by various media compression standards, e.g., MPEG-2, MPEG-4, H.323 and H.324. FIG. 1 illustrates scaling of multimedia content in accordance with conventional media compression standards. As illustrated in FIG. 1, these compression standards use the original multimedia content to generate a base layer and one or more enhancement layers. The base layer contains the minimum amount of information required to understand the multimedia content, while the enhancement layer provides information which can be used to enhanced the reproduction of the information in the base layer. Accordingly, a computer with a large processing capacity can reproduce the multimedia content using the base layer stream and one or more of the enhancement layer streams, while a computer with low processing capacity may reproduce the multimedia content using only the base layer information. These compression standards also support the ability to negotiate between different clients the amount of compression used on the multimedia content exchanged between the clients. However, these compression schemes generally scale the multimedia data down to the lowest common denominator, i.e., the quality used by all parties is based on the processing power of the slowest computer. One of the deficiencies of scalable video coding is the increased computational complexity required of the client producing the scalable video information. This is due to the fact that the client must produce as many layers as different preferences and clients exist in the network. Conventionally, a limited number of layers are generated which may cause problems in multi-party video conferencing applications.
Another method for scaling multimedia data is known as transcoding. Transcoding uses a transcoder to convert one type of compressed signal format into another type of compressed signal format. Accordingly, a transcoder can be used to scale multimedia down to a particular client's processing power and bandwidth in accordance with user preferences and client capabilities. However, video transcoding increases the complexity and delays in the network. The additional computational complexity and delays is a great disadvantage for real-time communication applications.
Known multimedia scaling methods may result in wasted bandwidth and processing power due to a lack of consideration of the ability of an end user to understand the received multimedia content. For example, although video information may provide a great amount of understanding of the content therein to a user with a computer, a user of a mobile phone which does not have the ability to process a video would have no understanding of the content therein. Accordingly, conventional scalable multimedia systems will transmit information which may not be able to be understood by an end user which results in wasted bandwidth.
Accordingly, it would be desirable to provide a multimedia scaling system which does not suffer from the deficiencies of known multimedia scaling systems.
Further, it would be desirable to provide a multimedia scaling system which does not limit all parties to a communication session to the capabilities of the slowest party.
In addition, it would be desirable to provide the proper combinations of media to an end user which provides the best understanding of the content therein based upon end user preferences and client device capabilities.